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ABSTRACT 

In this paper, we deal with some specific domains of ap- 
plications to game theory. This is one of the major class 
of models in the new approaches of modelling in the eco- 
nomic domain. For that, we use genetic automata which 
allow to buid adaptive strategies for the players. We ex- 
plain how the automata-based formalism proposed - ma- 
trix representation of automata with multiplicities - allows 
to define a semi-distance between the strategy behaviors. 
With that tools, we are able to generate an automatic pro- 
cessus to compute emergent systems of entities whose be- 
haviors are represented by these genetic automata. 



1. Introduction: Adaptive Behaviour Modeling for Ga- 
me Theory 

Since the five last decades, game theory has become a ma- 
jor aspect in economic sciences modelling and in a great 
number of domains where strategical aspects has to be in- 
volved. Game theory is usually defined as a mathematical 
tool allowing to analyse strategical interactions between 
individuals. 

Initially funded by mathematical researchers, J. von 
Neumann, E. Borel or E. Zermelo in 1920s, game theory 
increased in importance in the 1940s with a major work 
by J. von Neumann and O. Morgenstern and then with 
the works of John Nash in the 1950s |9|. John Nash has 
proposed an original equilibrium ruled by an adaptive cri- 
terium. In game theory, the Nash equilibrium is a kind of 
optimal strategy for games involving two or more players, 
whereby the players reach an outcome to mutual advan- 
tage. If there is a set of strategies for a game with the prop- 
erty that no player can benefit by changing his strategy 
while the other players keep their strategies unchanged, 
then this set of strategies and the corresponding payoffs 
constitute a Nash equilibrium. 

We can understand easily that the modelization of a 
player behavior needs some adaptive properties. The com- 
putable model corresponding to genetic automata are in 



this way a good tool to modelize such adaptive strategy. 

The plan of this paper is the following. In the next sec- 
tion, we present some efficient algebraic structures, the au- 
tomata with multiplicities, which allow to implement pow- 
erful operators. We present in section 3, some topological 
considerations about the definition of distances between 
automata which induces a theorem of convergence on the 
automata behaviors. Genetic operators are proposed for 
these automata in section 4. For that purpose, we show 
that the relevant "calculus" is done by matrix represen- 
tions unravelling then the powerful capabilities of such al- 
gebraic structures. In section 5, we focus our attention on 
the "iterated prisonner dilemma" and we buid an original 
evolutive probabilistic automaton for strategy modeling, 
showing that genetic automata are well-adapted to model 
adaptive strategies. Section 6 shows how we can use the 
genetic automata developed previously to represent agent 
evolving in complex systems description. An agent behav- 
ior semi-distance is then defined and allows to propose an 
automatic computation of emergent systems as a kind of 
self-organization detection. 

2. Automata from boolean to multiplicies theory (Au- 
tomata with scalars) 

Automata are initially considered as theoretical tools. They 
are created in the 1950's following the works of A. Tur- 
ing who previously deals with the definition of an abstract 
"machine". The aim of the Turing machines is to define 
the boundaries for what a computing machine could do 
and what it could not do. 

The first class of automata, called finite state automata 
corresponds to simple kinds of machines |21|. They are 
studied by a great number of researchers as abstract con- 
cepts for computable building. In this aspect, we can re- 
call the works of some linguist researchers, for example N. 
Chomsky who defined the study of formal grammars. 

In many works, finite automata are associated to a rec- 
ognizing operator which allows to describe a language 1 2 
1101 . In such works, the condition of a transition is simply 
a symbol taken from an alphabet. From a specific state S, 



the reading of a symbol a allows to make the transitions 
which are labeled by a and come fromS (in case of a 
deterministic automaton - a DFA - there is only one tran- 
sition - see below). A whole automaton is, in this way, 
associated to a language, the recognized language, which 
is a set of words. These recognized words are composed 
of the sequences of letters of the alphabet which allows to 
go from a specific state called initial state, to another spe- 
cific state, called final state. 

A first classification is based on the geometric aspect : 
DFA (Deterministic Finite Automata) and NFA (Nonde- 
terministic Finite Automata). 

• In Deterministic Finite Automata, for each state there 
is at most one transition for each possible input and 
only one initial state. 

• In Nondeterministic Finite Automata, there can be 
none or more than one transition from a given state 
for a given possible input. 

Besides the classical aspect of automata as machines 
allowing to recognize languages, another approach con- 
sists in associating to the automata a functional goal. In 
addition of accepted letter from an alphabet as the condi- 
tion of a transition, we add for each transition an infor- 
mation which can be considered as an output data of the 
transition, the read letter is now called input data. We de- 
fine in such a way an automaton with outputs or weighted 
automaton. 

Such automata with outputs give a new classification 
of machines. Transducers are such a kind of machines, 
they generate outputs based on a given input and/or a state 
using actions. They are currently used for control appli- 
cations. Moore machines are also such machines where 
output depends only on a state, i.e. the automaton uses 
only entry actions. The advantage of the Moore model is 
a simplification of the behaviour. 

Finally, we focus our attention on a special kind of au- 
tomata with outputs which are efficient in an operational 
way. This automata with output are called automata with 
multiplicities. An automaton with multiplicities is based 
on the fact that the output data of the automata with out- 
put belong to a specific algebraic structure, a semiring 
1131 1221 . In that way, we will be able to build effective 
operations on such automata, using the power of the alge- 
braic structures of the output data and we are also able to 
describe this automaton by means of a matrix representa- 
tion with all the power of the new (i.e. with semirings) 
linear algebra. 

Definition 1 (Automaton with multiplicities) 

An automaton with multiplicities over an alphabet A and 
a semiring K is the 5-uple {A, Q, I, T, F) where 

• Q = {Si, S2 ■ ■ ■ S n } is the finite set of state; 



• I : Q 1— * K is a function over the set of states, which 
associates to each initial state a value of K, called 
entry cost, and to non- initial state a zero value ; 

• F : Q 1— ► K is a function over the set states, which 
associates to each final state a value of K, called 
final cost, and to non-final state a zero value; 

• T is the transition function, that is T : Qx AxQ 1— > 
K which to a state Si, a letter a and a state Sj as- 
sociates a value z of K ( the cost of the transition) if 
it exist a transition labelled with a from the state Si 
to the state Sj and and zero otherwise. 

Remark 1 Automata with multiplicities are a generalisa- 
tion of finite automata. In fact, finite automata can be 
considered as automata with multiplicities in the semiring 
K, the boolan set B = {0, 1} (endowed with the logical 
"or/and"). To each transition we affect 1 if it exists and 
if not. 

Remark 2 We have not yet, on purpose, defined what a 
semiring is. Roughly it is the least structure which allows 
the matrix "calculus " with unit ( one can think of a ring 
without the "minus" operation). The previous automata 
with multiplicities can be, equivalently, expressed by a ma- 
trix representation which is a triplet 

• A G K lx Q which is a row-vector which coefficients 
are \ = I (Si), 

• 7 G K® xl is a column-vectorwhich coefficients are 

7* - F(Si), 

• p, : A* 1— » K® y Q is a morphism of monoids (in- 
deed KQ X Q is endowed with the product of matri- 
ces) such that the coefficient on the q^th row and qjth 
column of 11(a) is T(qi, a, qj) 

3. Topological considerations 

If K is a field, one sees that the space At n ) of automata 
of dimension n (with multiplicities in K) is a if -vector 
space of dimension k.n 2 + 2n (k is here the number of 
letters). So, in case the ground field is the field of real 
or complex numbers 0, one can take any vector norm 
(usually one takes one of the Holder norms ||(xi)i£i|| Q := 

( Yliel \ x i\ a ) " f° r a — 1> b ut an y norm will do) and the 
distance is derived, in the classical way, by 

d(Ai,A 2 ) = norm(V(Ax) - V(A 2 )) (1) 

where stands for the vector of all coefficients of 

A = (A, /1, 7) arranged in some order one has then the 
result of Theorem^ Assuming that K is the field of real or 
complex numbers, we endow the space of series/behaviours 
with the topology of pointwise convergence (Topology of 
F. Treves E3). 



Theorem 1 Let (A n ) be a sequence of automata with limit 
C (C is an automaton), then one has 

Behaviour (C) = lim Behaviour (A n ) (2) 

n — >oo 

where the limit is computed in the topology of Treves. 

4. Genetic automata as efficient operators 

We define the chromosome for each automata with multi- 
plicities as the sequence of all the matrices associated to 
each letter from the (linearly ordered) alphabet. The chro- 
mosomes are composed with alleles which are here the 
lines of the matrix (6). 

In the following, genetic algorithms are going to gener- 
ate new automata containing possibly new transitions from 
the ones included in the initial automata. 

The genetic algorithm over the population of automata 
with multiplicities follows a reproduction iteration broken 
up in three steps I14II18IIT71 : 

• Duplication: where each automaton generates a clone 
of itself; 

• Crossing-over, concerns a couple of automata. Over 
this couple, we consider a sequence of lines of each 
matrix for all. For each of these matrices, a permu- 
tation on the lines of the chosen sequence is made 
between the analogue matrices of this couple of au- 
tomata; 

• Mutation: where a line of each matrix is randomly 
chosen and a sequence of new values is given for 
this line. 

Finally the whole genetic algorithm scheduling for a 
full process of reproduction over all the population of au- 
tomata is the evolutionary algorithm: 

1. For all couple of automata, two children are cre- 
ated by duplication, crossover and mutation mech- 
anisms; 

2. The fitness for each automaton is computed; 

3. For all 4-uple composed of parents and children, the 
performless automata, in term of fitness computed 
in previous step, are suppressed. The two automata, 
still living, result from the evolution of the two ini- 
tial parents. 

Remark 3 The fitness is not defined at this level of ab- 
stract formulation, but it is defined corresponding to the 
context for which the automaton is a model, as we will do 
in the next section. 



5. Applications to competition-cooperation modeling 
using prisoner dilemma 

We develop in this section how we can modelize competition- 
cooperation processes in a same automata-based represen- 
tation. The genetic computation allows to make automatic 
transition from competition to cooperation or from coo- 
peartion to competition. The basic problem used for this 
purpose is the well-known prisoner dilemma [ 1 1. 

5.1. From adaptive strategies to probabilistic automata 

The prisoner dilemma is a two-players game where each 
player has two possible actions: cooperate (C) with its ad- 
versary or betray him (C). So, four outputs are possible 
for the global actions of the two players. A relative payoff 
is defined relatively to these possible outputs, as described 
in the following table where the rows correspond to one 
player behaviour and the columns to the other player one. 





c 


C 


c 


(3,3) 


(0,5) 


c 


(5,0) 


(1,1) 



Table 1. Prisoner dilemma payoff 

In the iterative version of the prisoner's dilemma, suc- 
cessive steps can be defined. Each player do not know 
the action of its adversary during the current step but he 
knows it for the preceding step. So, different strategies 
can be defined for a player behaviour, the goal of each one 
is to obtain maximal payoff for himself. 

In Figures ^ an d ID we describe two strategies with 
transducers. Each transition is labeled by the input cor- 
responding to the player perception which is the prece- 
dent adversary action and the output corresponding to the 
present player action. The only inital state is the state 1, 
recognizable by the incoming arrow labeled only by the 
output. The final states are the states 1 and 2, recognizable 
with the double circles. 

In the strategy of Figure ^ the player has systemati- 
cally the same behaviour as its adversary at the previous 
step. In the strategy of FigurefSJ the player chooses defini- 
tively to betray as soon as his adversary does it. The previ- 
ous automaton represents static strategies and so they are 
not well adapted for the modelization of evolutive strate- 
gies. For this purpose, we propose a model based on a 
probabilistic automaton described by Figurel3ll5l. 

This automaton represents all the two-states strategies 
for cooperation and competitive behaviour of one agent 
against another in prisoner's dilemma. 

The transitions are labeled in output by the probabili- 
ties pi of their realization. The first state is the state reached 



after cooperation action and the second state is reached af- 
ter betrayal. 



C:C 




C:C 

C:C C:C 

Figure 1. Tit-for-tat strategy automaton 



For this automaton, the associated matrix representa- 
tion, as described previously, is: 



F 

T{C) 
T{C) 



(pi l-Pi) 

P6 
1 -P6 

P2 1-P2 
P3 1 - P3 

Pi 1 - P4 
P5 1 - P5 



(3) 
(4) 

(5) 

(6) 




Figure 2. Vindictive strategy automaton 




l-pi 



Figure 3. Probabilistic multi-strategies two-states au- 
tomaton 



5.2. From probabilistic automata to genetic automata 

With the matrix representation of the automata, we can 
compute genetic automata as described in previous sec- 
tions. Here the chromosomes are the sequences of all the 
matrices associated to each letter. We have to define the 
fitness in the context of the use of these automata. The 
fitness here is the value of the payoff. 



5.3. General Genetic Algorithm Process for Genetic 
Automata 

A population of automata is initially generated. These au- 
tomata are playing against a predefined strategy, named 
S . 

Each automaton makes a set of plays. At each play, we 
run the probabilistic automaton which gives one of the two 
outputs: (C) or (C). With this output and the So's output, 
we compute the payoff of the automaton, according with 
the payoff table. 

At the end of the set of plays, the automaton payoff is 
the sum of all the payoffs of each play. This sum is the 
fitness of the automaton. At the end of this set of plays, 
each automaton has its own fitness and so the selection 
process can select the best automata. At the end of these 
selection process, we obtain a new generation of automata. 

This new generation of automata is the basis of a new 
computation of the 3 genetics operators. 

This processus allows to make evolve the player's be- 
havior which is modelized by the probabilistic multi-stra- 
tegies two-states automaton from cooperation to compe- 
tition or from competition to cooperation. The evolution 
of the strategy is the expression of an adaptive computa- 
tion. This leads us to use this formalism to implement 
some self-organisation processes which occurs in complex 
systems. 



6. Extension to Emergent Systems Modeling 

In this section, we study how evolutive automata-based 
modeling can be used to compute automatic emergent sys- 
tems. The emergent systems have to be understood in the 
meaning of complex system paradigm that we recall in the 
next section. We have previously defined some way to 
compute the distance between automata and we use these 
principles to define distance between agents behaviours 
that are modeled with automata. Finally, we defined a spe- 
cific fitness that allows to use genetic algorithms as a kind 
of reinforcement method which leads to emergent system 
computation 1 15 1. 

6.1. Complex System Description Using Automata-Ba- 
sed Agent Model 



According to General System Theory 141 1191 . a complex 
system is composed of entities in mutual interaction and 
interacting with the outside environment. A system has 
some characteristic properties which confer its structural 
aspects, as schematically described in part (a) of Figure^ 

• The set elements or entities are in interactive depen- 
dance. The alteration of only one entity or one in- 
teraction reverberates on the whole system. 

• A global organization emerges from interacting con- 
stitutive elements. This organization can be identi- 
fied and carries its own autonomous behavior while 
it is in relation and dependance with its environ- 
ment. The emergent organization possesses new prop- 
erties that its own constitutive entities don't have. 
"The whole is more than the sum of its parts". 

• The global organization retro-acts over its constitu- 
tive components. "The whole is less than the sum of 
its parts" after E. Morin. 



The interacting entities network as described in part 

(b) of Figure |4] leads each entity to perceive informations 
or actions from other entities or from the whole system 
and to act itself. 

A well-adapted modeling consists of using an agent- 
based representation which is composed of the entity called 
agent as an entity which perceives and acts on an environ- 
ment, using an autonomous behaviour as described in part 

(c) of Figure^ 

To compute a simulation composed of such entities, 
we need to describe the behaviour of each agent. This one 
can be schematically described using internal states and 
transition processes between these states, as described in 
part (d) of Figure|4] 

There are several definitions of "agents" or "intelligent 
agents" according to their behaviour specificities II II 1241 . 
Their autonomy means that the agents try to satisfy a goal 



and execute actions, optimizing a satisfaction function to 
reach it. 

For agents with high level autonomy, specific actions 
are realized even when no perception are detected from 
the environment. To represent the process of this delib- 
eration, different formalisms can be used and a behaviour 
decomposed in internal states is an effective approach. Fi- 
nally, when many agents operate, the social aspects must 
also be taken into account. These aspects are expressed 
as communications through agent organisation with mes- 
sage passing processes. Sending a message is an agent ac- 
tion and receiving a message is an agent perception. The 
previous description based on the couple: perception and 
action, is well adapted to this. 

6.2. Agent Behavior Semi-Distance 

We describe in this section the bases of the genetic algo- 
rithm used on the probabilistic automata allowing to man- 
age emergent self-organizations in the multi-agent simula- 
tion. 

For each agent, we define e an evaluation function of 
its own behaviour returning the matrix M of values such 
that Mi j is the output series from all possible successive 
perceptions when starting from the initial state i and end- 
ing at the final state j, without cycle. It will clearly be 
if either i is not an initial state or j is not a final one and 
the matrix M,j is indeed a matrix of evaluations |2| of 
subseries of 



M* 



(7) 



Notice that the coefficients of this matrix, as defined, 
are computed whatever the value of the perception in the 
alphabet A on each transition on the successful path 1 . That 
means that the contribution of the agent behaviour for col- 
lective organization formation is only based, here, on prob- 
abilities to reach a final state from an initial one. This al- 
lows to preserve individual characteristics in each agent 
behaviour even if the agent belongs to an organization. 

Let x and y two agents and e(x) and e(y) their respec- 
tive evaluations as described above. We define d(x, y) a 
semi-distance (or pseudometrics, see ch IX) between 
the two agents x and y as ||e(a;) — e(y)||, a matrix norm 
of the difference of their evaluations. Let V x a neighbour- 
hood of the agent x, relatively to a specific criterium, for 
example a spatial distance or linkage network. We define 
f{x) the agent fitness of the agent x as : 

™ r f x \ 2 if £ d(*.i«)Vo 
f(x) = { 1^ d{x,yt) wev* 

oo otherwise 



A succesful path is a path from an initial state to a final state 




(a) Global complex system (b) Interacting entities network 




(c) Agent-based model for entity (d) automata-based model for agent behavioui 

Figure 4. Multi-scale complex system description: from global to individual models 



6.3. Evolutive Automata for Automatic Emergence of 
Self-Organized Agent- Based Systems 

In the previous computation, we denned a semi-distance 
between two agents. This semi-distance is computed using 
the matrix representation of the automaton with multiplic- 
ities associated to the agent behaviour. This semi-distance 
is based on successful paths computation which needs to 
define initial and final states on the behaviour automata. 
For specific purposes, we can choose to define in some 
specific way, the initial and final states. This means that 
we try to compute some specific action sequences which 
are characterized by the way of going from some spe- 
cific states (defined here as initial ones) to some specific 
states (defined here as final ones). 

Based on this specific purpose which leads to define 
some initial and final states, we compute a behaviour semi- 
distance and then the fitness function defined previously. 
This fitness function is an indicator which returns high 
value when the evaluated agent is near, in the sense of 
the behaviour semi-distance defined previously, to all the 
other agents belonging to a predefined neighbouring. 



Genetic algorithms will compute in such a way to make 
evolve an agent population in a selective process. So dur- 
ing the computation, the genetic algorithm will make evolve 
the population towards a newer one with agents more and 
more adapted to the fitness. The new population will con- 
tain agents with better fitness, so the agents of a population 
will become nearer each others in order to improve their 
fitness. In that way, the genetic algorithm reinforces the 
creation of a system which aggregates agents with similar 
behaviors, in the specific way of the definition of initial 
and final states defined on the automata. 

The genetic algorithm proposed here can be consid- 
ered as a modelization of the feed-back of emergent sys- 
tems which leads to gather agents of similar behaviour, 
but these formations are dynamical and we cannot predict 
what will be the set of these aggregations which depends 
of the reaction of agents during the simulation. Moreover 
the genetic process has the effect of generating a feed- 
back of the emergent systems on their own contitutive ele- 
ments in the way that the fitness improvement lead to bring 
closer the agents which are picked up inside the emergent 



aggregations. 

For specific problem solving, we can consider that the 
previous fitness function can be composed with another 
specific one which is able to measure the capability of the 
agent to solve one problem. This composition of fitness 
functions leads to create emergent systems only for the 
ones of interest, that is, these systems are able to be de- 
veloped only if the aggregated agents are able to satisfy 
some problem solving evaluation. 

7. Conclusion 

The aim of this study is to develop a powerful algebraic 
structure to represent behaviors concerning cooperation- 
competition processes and on which we can add genetic 
operators. We have explained how we can use these struc- 
tures for modeling adaptive behaviors needed in game the- 
ory. More than for this application, we have described 
how we can use such adaptive computations to automat- 
ically detect emergent systems inside interacting networks 
of entities represented by agents in a simulation. 
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